Masked Autoencoders (MAE) play a pivotal role in learning potent
representations, delivering outstanding results across various 3D perception
tasks essential for autonomous driving. In real-world driving scenarios, it's
commonplace to deploy multiple sensors for comprehensive environment
perception. While integrating multi-modal features from these sensors can
produce rich and powerful features, there is a noticeable gap in MAE methods
addressing this integration. This research delves into multi-modal Masked
Autoencoders tailored for a unified representation space in autonomous driving,
aiming to pioneer a more efficient fusion of two distinct modalities. To
intricately marry the semantics inherent in images with the geometric
intricacies of LiDAR point clouds, the UniM$^2$AE is proposed. This model
stands as a potent yet straightforward, multi-modal self-supervised
pre-training framework, mainly consisting of two designs. First, it projects
the features from both modalities into a cohesive 3D volume space, ingeniously
expanded from the bird's eye view (BEV) to include the height dimension. The
extension makes it possible to back-project the informative features, obtained
by fusing features from both modalities, into their native modalities to
reconstruct the multiple masked inputs. Second, the Multi-modal 3D Interactive
Module (MMIM) is invoked to facilitate the efficient inter-modal interaction
during the interaction process. Extensive experiments conducted on the nuScenes
Dataset attest to the efficacy of UniM$^2$AE, indicating enhancements in 3D
object detection and BEV map segmentation by 1.2\%(NDS) and 6.5\% (mIoU),
respectively. Code is available at this https URL

该篇研究论文提出了一种用于自动驾驶的多模态蒙版自动编码器（UniM$^2$AE）模型，通过将图像与激光雷达点云的特征融合，实现了对多模态数据的高效处理，提高了三维物体检测和鸟瞰图分割的效果。